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INTRODUCTION 

In  1973  Markel  and  Gray  [  1 1  presented  a  geometrical  formulation  of  the  lattice 
approach  to  regression  analysis  (also  used  by  Itakura  and  Saito  [2] ).  Their  goal  was  the  so¬ 
lution  of  a  stochastic  problem  (least  mean  squares  or  so-called  autocorrelation  or  normal 
equations)  and  consequently,  the  relations  derived  were  functions  of  expected  values  of 
variables.  In  particular,  the  results  rely  on  the  true  autocorrelation  matrix.  More  recently, 
Morf  et  al.  ([3]-[8] )  have  developed  time-recursive  algorithms  for  the  solution  of  the  least 
squares  problem;  i.e.,  the  minimization  of  a  finite  time  average  of  errors  rather  than  the 
expectation  of  the  error.  The  least  squares  problem  also  involves  a  set  of  normal  equations, 
but  in  this  case  the  relevant  matrix  is  a  finite-data  approximation  to  the  autocorrelation 
matrix. 

In  this  paper  we  utilize  an  inner-product  formalism  similar  to  that  of  [  1  ]  to  solve  the 
least  squares  problem.  The  structure  so  obtained  is  of  considerable  interest  inasmuch  as  it 
forms  the  basis  for  a  whole  class  of  recursive  least  squares  lattice  algorithms  ([3 ]-[ 9] ).  This 
apporach  also  enjoys  several  other  advantages.  First,  it  provides  a  simplified  derivation  of  the 
above  mentioned  algorithms.  (It  is  especially  noteworthy  that  the  inversion  lemma  for  a  par¬ 
titioned  matrix  is  not  needed.)  Secondly,  the  geometrical  nature  of  the  variables  defined 
makes  their  role  in  the  algorithms  readily  apparent  and  lends  some  physical  intuition  as  well. 
Finally,  many  of  the  relations  proved  in  [  1  ]  for  expected  values  are  shown  to  hold  true  for 
the  actual  variables.  These  relations,  such  as  the  expression  of  the  lattice  gain  coefficients  in 
terms  of  inner  products,  can  provide  important  insights  into  the  algorithms’  behavior;  i.e., 
regarding  numerical  stability,  convergence  (effect  of  T-*»),  implementation,  etc.  ([1 J  ,[9] ). 


DEFINITIONS 

We  begin  with  some  vector  notation.  Let  x(t)  be  a  discrete  time  series,  and  define  x 
to  be  the  vector  whose  t^1  component  is  given  by 

[x]t  =  x(t)  t  =  0,  1 .  (1) 

Although  this  vector  lies  in  an  infinite  dimensional  space  (x  e  R°°),  all  our  operations  shall 
be  on  finite  dimensional  subspaces  thus  alleviating  the  need  for  sophisticated  mathematical 
techniques.  However,  if  so  desired,  the  reader  may  assume  an  upper  bound  Tq  such  that 
T 

0  <  t  <  Tq;  i.e.,  x  e  R  0.  By  shifting  x,  we  obtain  a  family  of  vectors  x1 

!x(t  —  i)  t  >  i 

(2) 

0  0<t<i 

This  family  generates  a  nested  set  of  subspaces  S'3  defined  by 


I 


where  the  curly  brackets  indicate  the  linear  space  spanned  by  the  vectors  it  encloses.  It  is 
assumed  that  the  xi’s  are  linearly  independent  for  0  <  i  <  Pq;  thus,  is  p  +  1  dimensional 
for  p  <  Pq.  Note  also  that  S^c  Sr  for  p  <  r. 

We  next  introduce  a  family  of  pseudo-metrics  on  R°°  defined  as  follows: 


T 

<x,  y>x  =  ^  wT_t[xlt  [yl  t 

t=0  (4) 

I  I  x  |  |2t  =  <x,  x>T 

where  0  <  w  <  1  is  constant  factor  which  “exponentially  windows”  the  data  ([8]  -[9] ). 
Although  <  >tt  is  singular  on  R°°,  it  is  a  true  metric  on  for  sufficiently  large  T.  More 
precisely,  a  necessary  and  sufficient  condition  for  <  >-p  to  be  a  metric  on  S  for  all 
p  <  Pq  is  that  the  truncated  vectors  x1  defined  by 


[x‘]t  t  < T 
0  t  >  T 


(5) 


be  linearly  independent  for  0  <  i  <  pQ  (see  Appendix).  We  shall  always  assume  this  to  be 
the  case.  Note  that  a  necessary  condition  is  T  >  Pq. 

It  is  also  convenient  to  define  the  linear  shift  operator  z1  by 


[z'x]t  = 


Wt+i  t  +  i>0 
0  t  +  i  <  0 


(6) 


where  i  may  be  any  integer.  It  follows  that 

z_i  x  =  x*  .  (7) 

Note  that  zz-^  =  Ibutz“*z#  I  where  I  is  the  identity.  A  short  calculation  utilizing  def¬ 
initions  (6)  and  (4)  yields  the  relation 

<z_1  x,  y  )T  =  <x,  zy  >T_j  .  (8) 

Equation  (8)  will  prove  extremely  important  inasmuch  as  it  relates  time  shift  properties  to 
order  (i.e.,  to  the  index  i  of  the  family  xl).  It  differs  from  the  stochastic  case  of  Markel 
and  Gray  1 1  ]  inasmuch  as  z  and  z~  *  are  not  quite  adjoint  operators  (T  is  replaced  by  T-l 
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in  (8) ).  However,  the  shift  properties  are  sufficiently  powerful  to  allow  <  >  to  replace 
expectation  in  the  formalism. 

The  least  squares  problem  whose  solution  we  are  ultimately  seeking  is  the  following: 
Given  sequences  x(t)  and  y(t),  find  gP  such  that 


iy  -  ^  gP  X1  1  1^.  =  min  1  ly  -  ^  qjX1  |  1^  . 


Actually  this  is  a  family  of  problems  where  the  order  p  may  range  over  0< p  < pg  and  the 
time  T  over  pg  <  T  <  Tg.  Note  that  the  weights  w  appearing  in  the  definition  of  the  metric, 
(4),  may  be  thought  of  as  simply  error  weights,  or  as  a  sliding  exponential  window 
(x(t)  ♦  (Vw)  x  (t))  of  the  data  [9] . 


LATTICE  ORDER  RECURSION  RELATIONS 

We  shall  repeatedly  have  need  of  the  phrase  “orthogonal  projection  with  respect  to 
the  metric  <  >  j,”  and  hence  shall  use  the  abbreviation  “projj.”  Let 
P-1 

bP (T)  x‘  be  the  proj-j.  of  xp  onto  Sp_*  (where  S'*  =  empty  set).  Then  we 
i=0 

define  0P  (T)  by 


0P(T)  =  xp  -  b?  (T)  x  *  .  (10) 

i % 

These  vectors  have  several  important  properties. 

We  first  note  that  0P(T)  is  the  projj  of  xp  onto  the  orthogonal  complement  of  SP-*  in 
Sp;i.e.,  (T),  xp  >T  =  <  (SP-1)1,  xp  >j.  Also  Sp  =  Sp-1  ©  0P  so  that  f  for  0  <  i  ^  p 
form  an  orthogonal  basis  of  Sp.  In  physical  terms  we  may  consider  0P  (T)  as  the  error  resid¬ 
ual  of  a  least  squares  backward  predictor  (estimator  of  xp  by  x*’s  for  i  <  p)  since 


1  ixp  -  ^  bP  (T)  x*  I  !2  =  min  I  |xp  -  £  qjx‘|  |2  . 
i=0  i=0 


For  future  convenience,  we  define  the  magnitude-squared  of  jr  (T)  by 


(T)  =  I  1 0P  (T)  i  lT2 
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It  follows  from  the  definitionof  jjP  (T)  that  it  is  uniquely  determined  by  the  three 
conditions 


0P(T)  eSp 

(13a) 

0P  (T)  -  xp  e 

sp-l 

(13b) 

<0P(T),  xhT  = 

=  0  for  0  <  i  <  p-1  . 

(13c) 

Equations  ( 1 3)  merely  state  that  the  set  of  0*  (T)  is  obtained  by  applying  the  Gram- 
Schmidt  orthogonalization  procedure  to  x1,  0  <  i  <  p  with  respect  to  the  metric  <  >j. 

As  in  [  1  ]  we  introduce  a  set  of  auxiliary  vectors  ap  (T)  to  aid  in  the  order  recur¬ 
sion.  Let  Qp  be  the  subspace 


0"=  {** . *p  }• 

P 

and  let  a?  (T)  x*  be  the  projj  of  x°  onto  Qp.  Then  we  define  ap(T)  by 
i=  1 

P 

ap  (T)  =  *°  -  aP  x* 
i=1 


(14) 


(15) 


with  magnitude  squared 


ep(T)  =  1 1 5P  (T)  |  lT2  . 


(16) 


These  ap(T)  represent  forward  predictor  residuals  and  are  uniquely  determined  by  the  con¬ 
ditions 


5p(T)  e  Sp  (17a) 

5p(T)  -x°  e  QP*  (17b) 

<ap(T),  x‘>T  =  0  for  1  <i  <p  .  (17c) 

We  now  show  that  5P(T)  and  0P(T*-1 )  may  be  computed  recursively  via  the  rela¬ 


tions 


5p+1(T)  =  5P(T)  +  KP(T)  z"1  0P(T-1) 


(18) 


4 


(19) 


0P+1(T)  =  z_1jjP(T-l)  +  KP(T)  5P(T) 

where  KP(T)  and  K^(T)  are  constants  which  we  shall  subsequently  determine  (see  (26) 
and  (29)). 

To  prove  (18)  we  note  that  z”^0P (T-l)  e  QP+*  c  Sp+* ,  and  from  (8) 
<z~l^p(T-l),  x‘>T  =  (Fa-D.zx*)^,  for  i  >  0 
=  <^P(T-1),J5"1>t_i 

=  0  for  1  <i<p  .  (20) 

Thus  for  any  constant  K,  $  ^ap(T)  +  K  z-*/jP(T-l)  satisfies  ( 1 7a)  and  ( 1 7b)  with  p  re¬ 
placed  by  p  +  1 ,  and  also  satisfies 

<0,  x'>x  =  0  for  1  <i<p  .  (21) 

In  consideration  of  (21 ),  condition  ( 1 7c)  will  be  satisfied  for  p  +  1  provided 
K  is  chosen  such  that 

0=  <<?,xP+1>T  =  <  Sp(T)  +  Kz-1/3p(T-l),xp+1  >T  ; 


i.e.,  if  we  choose  K  =  KP(T)  where 


<aP(T),  xP+1  (T))x 

irP  /T)  - - 

“  <z~'  0P(T-l),xp+1  (T)>x 


(22) 


A  similar  analysis  verifies  (19)  where  K^(T)  is  chosen  to  satisfy 


(T)  =  - 


<z_10p  (T- 1 ),  x° >x 
<5p  (T),  x®>x 


(23) 


The  appearance  of  T-l  in  the  above  equations  should  be  noted.  The  z~'  was  necessary  in 
order  to  obtain  a  vector  whose  “xp+' coefficient”  was  non-zero  (see  eq.  (22))  which  in  turn 
necessitated  T-l  in  order  that  eq.  (20)  be  valid  (c.f.  remarks  after  eq.  (8)). 
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SIMPLIFICATION  OF  Kp  AND  Kp 

Equations  (22)  and  (23)  may  be  rewritten  in  a  form  which  provides  a  simple  geo¬ 
metric  interpretation.  We  first  note  that  since  (3  P  (T-l )  is  a  projj_ j  of  xP,  we  have 
(0P  (T-l),  xP  >x_j  =  (^(T-DjPa-D^.j  =  eP(T-l).  Thus 

(z-^Pa-D.xP+bj  =  <^P(T-1),xp>t_j  =  ejj  (T-l)  .  (24) 

Similarly 

<ap  (T),  x°>T  =  ep(T)  .  (25) 

1  a 

We  now  show  that  the  numerators  of  (22)  and  (23)  are  both  given  by 
kp(T)  4  <5p(T),z''1i3P(T-l)>x  •  (26) 

We  have,  using  (20), 


<ap(T),z_1/3P(T-l)>T  *  <x°  -  ^  a?  (T)x1,  z-1/3P  (T— 1  )>T 

i=l 


=  <x°,  z_I0P  (T—  1  )>T 


(27) 


and  from  (17c) 

P-1 

<5P(T),z_10P(T-l)>T  =  <5p(T),z"!  (xP  -  ^  b?(T-l)x')>T 

i=0 

=  <ap(T),  z~l  xp>T 

/ 

=  <ap  (T),  xp+1  >T  . 

It  thus  follows  from  (22),  (23)  and  (26)-(28)  that 


(28) 


Kp(T)  = 


Kp(T)  = 


_  kP  (T) 
eP(T-l) 

_  kP(T) 
%p(T) 


(29) 
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For  ease  of  notation  let  5  =  ap(T)and0  =  z“*0P(T-l).  Then 


Ka  =  - 


-(ocJ)j 


0  11 


and  Kp  = 


■  (a,  0  >j 

15  1|2 


(30) 


It  follows  from  the  Cauchy-Schwartz  inequality  that  their  product  satisfies 

0  <  KaK0  <  1  .  (31) 

This  condition  is  weaker  than  that  of  [11  where  the  gains  are  individually  less  than  one. 
However,  relations  (30)  are  exact  for  arbitrary  input  x(t)  whereas  [  1 1  assumes  a  knowledge 
of  the  true  correlation  matrix,  a  cpndition  rarely  met  in  practice.  For  w  =  1  and  under 
reasonable  conditions  on  x,  <x\  x-i  >f  =  Ry  approaches  the  true  correlation  matrix  as 
T  -»  ooso  that  lim|  Ka(T)l  <  1  and  lim  I  lCg(T)l  <  1 .  This  is  not  true  for  w  <  1 ,  and  either 

T-*x>  t~h>o  ^ 

I  Ka(T)  I  or  I  K^(T)  I  (but  not  both)  may  be  greater  than  one.  This  may  have  serious  stability 
consequences  ([  1  ]  -  [2] ,  [9] ). 


TIME  UPDATES 

We  introduce  a  set  of  auxiliary  vectors  |P  to  aid  in  the  time  update  defined  as  follows 
Let  e^  be  the  vector  representing  the  T**1  coordinate  axis;  i.e., 

|e~T|t  =  0  {Ut  ■  «2» 

Then  f p  (T)  is  defined  as  the  projj  of  e^  onto  Sp 


IP(T)  e  Sp 
( ip  (T),  y)j  =  <  e^,  y  >j 

=  [y ]  j  fory  e  Sp  . 


(33a) 


(33b) 


p 

(We  could  characterize  \ p  (T)  =  ^  cP  (T)  x 1  as  in  equations  ( 1 0)  -  ( 1 1 ),  but  this  is  not 

l K) 

needed  in  the  development.)  Thus,  f p  (T)  picks  out  the  T1*1  coordinate  of  all  vectors  y  in  Sp. 
It  is  worth  observing  that  |p  is  a  measure  of  the  influence  of  the  most  recent  data  point 
(coordinate  axis  e^)  on  the  p1*1  order  model  (subspace  Sp). 
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We  define  the  squared  magnitude  of  |p  (T)  to  be  <jP  (T) 

0P  (T)  =  I  I  f  P  (T)  I  i  j  .  (34) 

An  important  property  of  aP  (T)  is  that 


0  <  aP(T)  <  aq(T)  <  1  for  p  <  q  .  (35) 

Relation  (35)  follows  from  definition  (34)  and  the  nesting  of  the  SP’s:  SP  C  Sq  so  that  the 
magnitude  of  the  projections  f  P  must  be  an  increasing  function  of  p.  Also 

[  f p  (T)  ]  T  =  oP(T)  =  I  lfp  (T)  I  !  j  <  tfP(T)]^  which  implies  [|P(T)]X  <  1. 

Since  SP+^  =  SP(+)^P+*  (T)  and  the  set  of  vectors  (3*  (T)  /  yj ^  (T)  for 

0  <  i  <  p  +  1  form  an  orthonormalj  basis  of  Sp+^ ,  the  projj  of  onto  SP+  *  is  equal 
to  its  projection  onto  Sp  plus  the  vector 


<eT,  0P+1  (T)  /  J£ p+1  (T)  >T  (3p+i  (T)  /  Je p+1  (T)  .  Thus, 


<eT,  /3P+1  (T)>t 

fP+1  (T)  =  fP  (T)  +  - — - i-  p+1  (T) 

eP+1  (T) 


=  fP  (T)  + 


[F+1  (T)]j 


ejj+1  (T) 


/3P+1  (T) 


(36) 


This  gives  an  order  recursion  for  £p  (T)  and  aP  (T). 

We  now  derive  the  time  update  recursion  for  0P  .  A  useful  relationship  which  follows 
immediately  from  definition  (4)  is 

<x,  y>T  =  w  <x,  y>T_j  +  [x]y  [y]j  .  (37) 
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<0P(T-l),x'>T  =  w  <0P  (T-l,  x 1  >T_ !  +  [0P(T-1)JT  [xhx 
=  [0P(T-1)1T  [x^t  0<i<p-l 

=  [^Tp  (T-l )]T  <fp_1  (T)x‘>t  •  (38) 

Consider  0  £  0P(T-1)  -  [0P(T-1)1T  fP"1  (T).  Then,  from  (38), (0,  x{)T  =  0  for 
0<i<p-l.  Also  0  -  xp  e  SP~^  and  it  follows  that  0  satisfies  (13a)  -  (13c);  i.e., 

0  -  (3p  (T)  and 

0P  (T)  =  Pp  (T-l)  -  I0P  (T—  1 )  1 T  I15-1  (T)  .  (39) 

A  similar  calculation  (one  must  use  (8)  because  of  z-^ )  yields  the  time  update  for  ap 

ap  (T)  =  ap  (T-l )  -  [5P(T-1)1T  z'1  IP_I  (T-l)  .  (40) 

The  time  update  for  kp  easily  follows  from  equations  (26),  (28)  and  (40)  by  utilizing 
the  identities  (8)  and  (37). 


-  kp(T)  =  -  <ap(T),  xp+1>T 


=  -  (ap  (T-l ),  xp+1)T  +  [«p  (T-l )]T  (z-1  F'1  (T-l),  xp+,)T 

=  -  wkp  (T-l)  -  [5p(T-l)]T  [xp+11t 

+  [Sp  (T-l)ly  (fP-1  (T-l),  xp>T_!  .  (41) 

It  follows  from  (36)  (or  simple  geometric  considerations)  that 

<ip_l  (T-l ),  xp>T_i  =  <|p  (T-l ),  xp>T_i  -  -f -~T  -■ -T-  <Pp  (T-l ),  xp)T_1 

ep  (T-l) 


=  (  xp]y_i  -  (0P  (T-l)l y_j 


This  combined  with  (41 )  yields 


kP(T)  =  wkP(T-l)  +  [5P  (T— 1 ) ] T  lPp  (T— 1 ) lx_j  • 


(42) 


Two  useful  relations  which  may  be  used  in  deriving  various  forms  of  the  least  squares 

algorithm  ([91 )  are  obtained  by  taking  the  scalar  product j  of  (39)  and  (40)  with  £P  (T) 

[?P(T)]X  =  [0P(T-1)]t  (1  -  OP"1  (T))  (43a) 

[«P(T)]T  =  [5p(T-l)lT  (1  -  oP’1  (T— 1 ))  (43b) 


where  we  have  used  (36)  in  deriving  (43a). 

The  time  updates  for  £  P,  ep  ,  and  ep  rely  on  the  following  lemmas: 

0£  p 

(2-l  |P-1  (X-l ),  x^>T  =  (z_I  | P"1  (T-l),  x°  -  5p(T)>T 

=  (F'1  (T-l),  z  (x°  -  5p  (T)))^! 

=  [x°-ap(T)]T  (from  (36)) 

=  (x°]x  -  [ «p (T) ] T 

and  similarly 

(fP"1  (T),  xp)T  =  [xp]T  -  [  Jp  (T)  1  x  . 

We  can  now  calculate  |p  (T)  from  fP(T-l).  Let  <t>  -  z~ *  £p-'  (T-l)  +  Kap(T). 
Then  <0 ,  x‘>x  =  [xMX  forp>i>0.  For  i  =  0  we  determine  K  by 

(  X®  JX  ®  (0,  X^)-j- 

=  < r 1  p-1  (T-l),  X°  >x  +  K  ( ap  (T),  x°  >x  . 


(44a) 


(44b) 


Using  (44a)  we  have 


K  = 


[5p  (T)lx 

<T> 
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Thus, 


,  [aP/Tll-T 

Pm  =  i*1  |p_1  (T-n  +  — — I  sp(T> 

«£rn 


-  z'1  P(T-l)  - 


(Jp(T-1)IT-|  „  [5P(T)1T  . 


eP  (T-l) 


?P(T-1)  + 


«£<d 


ap  (T)  (45) 


where  we  have  used  (36). 

Also,  combining  (44a)  with  (40),  we  have 


ep  (T)  =  <5P  (T),  x°)T 

=  <ap  (T-l),  x°>T  -  [ap  (T— 1 )  1 T  <z_1  fp_1  (T-l),x°>T 
=  w  <ap  (T-l),  x°>T_i  +  [5P(T-1)]t  ([x°1t  -  (z"1  Ip_1  (T-l),x°>T) 
=  w  ep  (T-l)  +  (ap  (T—  1 ) I T  [«p  (T) I T  . 


(46) 


Finally,  it  follows  from  (44b)  that 

eP 

0 


ep(T)  =  weP(T-l)  +  l/3p(T-l)lT  1/3  p  (T)  ]  T  . 


(47) 


SOLUTION  OF  LEAST  SQUARES  PROBLEM 
The  least  squares  problem,  (9),  may  now  easily  be  solved  by  noting  that 


r 

gP  (T)  x‘  is  simply  the  projy  of  y  onto  Sp.  We  define 
i=0 


r 

7P  (T)  =  y  -  ^  g?  (T)  x*  . 
i=0 

The  by  now  familiar  procedure  yields  the  order  update  for  7P  in  the  form 


(48) 


7P+I  (T)  =  7P  <T)  +  Kp+1  (T)  jjp+1  (T) 


(49) 
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where 


KP+1  (T) 

kP+1  (T) 
eP+1  (T) 

(50) 

kp+l  (T) 

=  <7P(T),xP+1>t 

7 

(51) 

=  (7P  (T),  z-1  jjp  (T-l  )>T  . 

Similarly, 

the  time  updates  for  7P  and 

k|J  are  found  to  be 

7P (T)  = 

7P  (T-l )  -  [7P(T-1)1t$p 

(T) 

(52) 

kp  (T)  = 

wkP(T-l)  +  [7P_1  (T-l)l 

[0p(T)]t 

(53) 

with 


i* 


4 


[7P(T)]x  =  [7P(T-1)1t  (l-oP(T))  . 


(54) 


The  output  variables  we  seek  are  usually  [>P  (T— 1 )  ]  x  as  in  noise  cancelling,  or 
zP  (T)  =  y  (T)  -  (7P  (T-l  )]x  as  in  equalization  [9] .  (We  address  the  determination  of 
g?(T)  in  the  next  section.)  To  achieve  this,  our  algorithm  need  consider  only  a  single  com¬ 
ponent  of  the  vectors  ap,  |3  P,  and  7P.  More  specifically,  we  choose 


«P(T)  =  (ap  (T)]j 

(55a) 

0P(T)  =  (0p(T)]t 

(55b) 

7P(T)  =  (7P(T-1)]t 

(55c) 

where  removal  of  the  bar  indicates  a  particular  vector  component.  We  remark  that  relations 
(43)  and  (54)  allow  us  to  alternate  between  [  •  (T-l ) )  j  and  [  *  (T)  ]  x,  but  for  the  applica¬ 
tions  mentioned  (55c)  is  the  most  prudent  choice  [9] . 

The  algorithm  is  now  obtained  by  taking  the  T**1  component  of  (18)  and  (19)  and  the 
T  +  1st  of  (49).  Order  updates  for  the  e’s  are  obtained  by  taking  the  scalar  product  of  (18), 
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' 


(19)  and  (49)  with  ap  (T),  z“l  j3p  (T),  and  7P  (T),  respectively.  These,  combined  with  the 
time  updates  of  the  k’s  ((42)  and  (53))  and  relations  (29)  and  (37),  yield  the  algorithm  of 
reference  [9] 


a0  (T)  =  0°  (T)  =  x  (T) 

(56a) 

4(T)  =  <$(T)  =  weg(T-l)  +  x2(T) 

(56b) 

a-1 (T)  =  0 

(56c) 

7_1  (T)  =  y  (T)  . 

(56d) 

=  0, . . . ,  Po 

kp  (T)  =  wkp  (T-l)  +  — 

1  -  ap~‘  (T-l) 

(57a) 

ap+l  (X)  =  aP  (X)  _  pP  (T_i) 

ep  (T~l ) 

(57b) 

pP+l  (X)  =  pP(T-l)  -  (T-'  «p(T) 

e%(T) 

(57c) 

ep+1  (T)  =  ep(T)  -  (kp  (T))2/eP  (T-l) 

(57d) 

ep+1  (T)  =  ejj  (T-l )  -  (kp  (T))2/ep  (T) 

(57e) 

ap(T)  =  ap->  (T)  +  (0P  (T))2/ep  (T) 

(570 

•  kp  (T-l )  pp  (X) 

yp  (X)  =  7P-1  (T)  -  J  - 

eg  (T-l)  1  -  ap_1  (T) 

(57g) 

kp(T)  =  wkp (T-l)  +  7P_1  (T)  Pp( T)  . 

(57h) 

Note  that  (56b)  follows  from  (37).  Aspects  of  the  implementation  of  the  above  algorithm 
such  as  initialization  may  be  found  in  [9] . 
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THE  NATURAL  ISOMORPHISM 

It  is  sometimes  desirable  to  obtain  the  filter  coefficients  gP  (T)  (as  in  the  fast 
Kalman  algorithm,  141—15] ).  This  may  be  achieved  via  a  natural  isomorphism  which  is  the 
analogue  of  that  implicit  in  1 1 1 .  The  only  difference  is  that  here  the  inner  product  is  defined 
with  respect  to  the  estimated  correlation  matrix  rather  than  the  true  correlation  matrix. 

More  precisely,  consider  the  set  of  polynomials  in  z~*  of  the  form 


oo 

A(z-1)  =  £  ajz-i  (58) 

i=0 

where  it  is  assumed  that  A  has  finite  degree;  i.e.,  aj  =  0  for  i  >  p  for  some  p  <  °°.  Define  a 
family  of  inner  products  (*,  *)t  on  these  polynomials  by 

(A(z*! ),  B(z-1  )>r  =  J  ajRjj  (T)bj  (59) 

ij 

where 

T 

Rij(T)  =  ^  w1"*  x(t-i)  x(t-j)  .  (60) 

t=0 


Note  that  the  conditions  of  the  Appendix  insure  that  (*,  *)j  is  non-singular  on  all  polynomials 
of  degree  less  than  pg. 

A(z)  may  be  thought  of  as  an  operator  on  the  vector  space  of  equation  ( 1 )  and  the 
mapping  A(z-1 )  -*■  A(z-^  )x  maps  the  polynomials  of  degree  p  onto  Sp.  It  is  easy  to  see  that 
this  mapping  is  a  metric  space  isomorphism  (for  p<Pg). 

(A(z-1),  B(z_1))T  =  < A(z_1)x,  B(z-1  x)>T  .  (61) 


It  is  a  simple  matter  to  translate  the  recursions  of  the  previous  sections  into  recur¬ 
sions  of  polynomial  coefficients  via  the  inverse  isometry.  For  example,  let 
P 

$  p  (T)  =  cj*  (T)x1.  Since  the  x1  form  a  basis  for  Sp,  by  matching  the  coefficients  of 


i=0 


x1  on  both  sides  of  equation  (39),  we  obtain 


bP(T)  =  b?  (T— I )  -(P(T-1)]t  cp-1(T-l)  0<i<p-I. 


(62) 


14 


Similarly,  since  {y,  x1; i  =  1,  .  .  .  ,  p  }  form  a  basis  for  Sp  © y,  equation  (52)  yields  the 
filter  coefficients 

g?  (T)  =  g?  (T-l)  -  ItP"1  (T-1)1t  c? (T) .  (63) 

This  approach,  the  development  of  the  algorithm  through  the  polynomial  coefficients 
(aP,bP,cP,gP)  is  most  practical  if  the  order  is  fixed  at  some  value  p  =  pg,  and  the  time  up¬ 
dates  are  used  (c.f.  [5] ).  In  this  case  the  computational  burden  is  0(pg)  per  time  update.  If 
gP  for  0  <  p  <  pg,  and  0  <  i  <  p  is  required  there  are  p^/2  variables  to  evaluate,  and  the 
number  of  computations  becomes  O(Pg^).  Note,  also,  that  for  fixed  P  =  Pg  the  expressions 
such  as  (T— 1 )  ]  -p  which  appear  throughout  (as  in  (62))  may  be  computed  directly  from 
their  definitions  in  0(pg)  operations.  For  example, 

po-i 

(0PO  (T-l  )1T  =  x(T-pg)  -  bPO  (T-l )  x(T-i). 

i=0 

CONCLUDING  REMARKS 

In  this  paper  we  defined  a  class  of  metric  spaces  which  resulted  in  a  geometrical  deri¬ 
vation  of  least  squares  lattice  algorithms.  This  inner-product  formalism  is  closely  related  to 
that  found  in  [  1 1 ,  but  involves  the  actual  input  data  rather  than  its  second  order  statistical 
properties.  This  is  reflected  in  the  use  of  a  set  of  metrics  indexed  by  time  to  replace  the  ex¬ 
pected  value.  Note  that  such  an  approach  lends  itself  naturally  to  a  time-recursive  formulation. 

An  advantage  of  the  above  structure  is  that  it  provides  a  single  framework  which  en¬ 
compasses  an  entire  family  of  fairly  complex  algorithms  ([3] -[9] ).  Its  geometrical  nature 
also  provides  a  guide  for  the  intuition  which  should  be  of  use  in  implementation  and  future 
investigations.  Finally,  we  wish  to  point  out  that  there  remain  many  questions  which  were 
beyond  the  scope  of  this  paper.  Among  these  are  stability  properties,  the  role  of  w  for 
w  <  1 ,  the  significance  of  the  scalar  gains  K  and  a,  and  the  spectral  properties  of  the  lattice 
decomposition.  Some  of  the  present  techniques  may  also  generalize  to  the  case  treated 
in  [10]. 


APPENDIX 


INDEPENDENCE  ASSUMPTION 


The  metric  <  >'p  is  singular  on  Sp  *3  r^  not  all  zero  and 


such  that 


T 

0  °  ^  wT-t  t 
t=0 


0  «*•  [ y ] t  =  0  for  t  <  T 


./  •/  •  » 

q  x1  =  0  «*■  the  x1  are  linearly  dependent.  Note  that  x1  are  defined  in  (5). 

Note  also  <  >f  is  singular  3  rj  not  all  zero  such  that 
rj  x* ,  xJ  >-p  =  0  for  0  <  j  <  p  •*>•  The  matrix  Rp  (T)  defined  by  (60)  with  i, 
P 


j  <  p  satisfies  ^  RP.  (T)  rj  =  0  *>  Rp  (T)  is  singular. 

i=0 


The  independence  condition  on  the  vectors  x’  of  (5)  is  thus  a  mixing  condition.  It 
says  that  by  the  time  T  all  the  modes  of  the  estimated  correlation  matrix  Rp0  mUst  be 
excited;  i.e.,  Rp°  (T)  is  nonsingular. 
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